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The advent of problem-based assessment has opened the question of whether or not conventional 
approaches to assessment are truly a representative of students’ knowledge (Lampert, 2001). In 
an attempt to find out about the suitability of problem-based tests in evaluating student-teachers’ 
knowledge of teaching English, a problem-based test was designed and administered to 33 MA 
students who were selected through the quota approach at Payame Noor University. Comparison 
of the results of the problem-based test and conventional test through paired-samples t-test 
revealed that teacher-students gained higher scores in the conventional tests; however, the results 
of the semi-structured interview with the participants revealed that they considered problem- 
based test a true measure of their knowledge and favored it over conventional assessment due to 
its relevance to content and context of the course, its motivating nature, and its dynamic structure. 
This study can have pedagogical implications for language teachers and material developers. 
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INTRODUCTION 

Assessment is in no certain terms a significant aspect of edu¬ 
cation so much that Farhady, Jafarpoor, and Birjandi (1994) 
pointed out that successful education is incomplete with¬ 
out proper assessment. Recent approaches to learning and 
assessment such as problem-based learning have aimed at 
solving problems which had dominated traditional and con¬ 
ventional forms of education (Savery, 2006). Conventional 
forms of education as stated by Boud and Feletti (1997) tar¬ 
geted students’ memory rather than comprehension; there¬ 
fore, retention of newly learned materials was in spectrum. 
In line with conventional teaching, conventional assessment 
evaluated students in terms of what they remembered rather 
than what they could perform (Ansarian, Adlipour, Saber, & 
Shafei, 2016). Problem-based learning (PBL) as an innova¬ 
tive approach to learning stood against traditional approach¬ 
es to learning which harshly shoved down a load of content 
to the throats of the students (Kwan & Lee, 2009). Consis¬ 
tent with PBL, problem-based assessment attempted to mea¬ 
sure what students could perform rather than what they had 
memorized for a short period of time. 

Performance rather than competence is of prime signifi¬ 
cance to student-teachers. In this study, student-teachers are 
students at postgraduate level who are getting prepared for 
teaching at educational systems in future. Based on the reg¬ 
ulations of Iranian universities, these students should pass 
a course entitled ‘Practical Teaching’ to qualify for teach¬ 
ing English at the Iranian educational system. Through the 


course, the students should demonstrate their ability to han¬ 
dle authentic teaching issues that occur in language classes. 
However, whether the structure of the course per se leads the 
student-teachers to comprehension rather than memorization 
has not been delved into. Therefore, the researchers were 
motivated to compare the performance of these students in 
terms of traditional assessment and problem-based assess¬ 
ment. The students were also interviewed for their percep¬ 
tion about problem-based assessment. 

Assessment is a delicate concept in education. It cawn 
determine the future of the individuals and give direction to 
their goals. Therefore, it is of prime significance to assess 
students as precisely as possible. However, there seems to be 
a problem with the approach adopted to assess student-teach¬ 
ers’ teaching skill. The student-teachers’ performance is con¬ 
ventionally measured through giving lectures about teach¬ 
ing methods and taking multiple-choice-item tests, among 
others. It has already been proved that multiple-choice-item 
tests are not suitable for assessing one’s ability to solve prob¬ 
lems, and considering that practical teaching in language 
classes is replete with occasions where teachers should be 
able to think of a way to solve problems; therefore, the valid¬ 
ity of current testing system is under question. Inappropriate 
tests, as they seem to be, may lead to selecting teachers who 
are not capable of teaching language classes; therefore, the 
new generation of language learners may be affected. As this 
issue requires immediate attention and action, this study was 
conducted. 
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Purpose of the Study 

This study aimed at investigating the issue of assessment in 
the course ‘Practical Teaching’ from two different perspec¬ 
tives. Firstly, by measuring student-teachers’ current perfor¬ 
mance through a multiple-choice test and then by comparing 
it to their performance in problem-based tests. Secondly, 
the study aimed at delving into student-teachers perception 
about both conventional assessment and problem-based as¬ 
sessment to find out which is more suitable for them. 

Research Questions 

The following research questions were proposed for this study. 

1. Is there any significant difference between the stu¬ 
dent-teachers’ performance on conventional university 
test and problem-based tests with regard to the course 
‘Practical Teaching’? 

2. What is the perception of student-teachers about prob¬ 
lem-based tests for the course “Practical Teaching”? 

Significance of the Study 

It is very significant to select students as teachers who are 
indeed capable of delivering effective education. This would 
not be possible without proper assessment. In addition, it is 
important to hear the voice of young generation of teachers 
to find out what they think about the merits and demerits of 
conventional and problem-based assessment. This can help 
teachers to prepare more effective tests for the students. 

REVIEW OF THE LITERATURE 

In general, problem-based learning has a long history in edu¬ 
cation. Its pioneers at McMaster University in Canada began 
implementing this approach in 1970s (Hmelo-Silver, 2004). 
Primarily, problem-based learning belongs to medical edu¬ 
cation though it was soon adopted by many educational sys¬ 
tems with slight alternations. By definition, problem-based 
learning is an innovative and collaborative approach to 
learning in which students attempt to learn content and 
master learning skills (Hmelo-Silver, 2004). Students learn 
content by making use of their higher order thinking skills 
(Saver, 2006), a process which often leads to retention rather 
than memorization. 

One of the main underlying assumptions about prob¬ 
lem-based learning was that the materials learnt through 
higher order thinking skills have a higher chance of reten¬ 
tion. Boud and Fletti (1991) assumed that 90% of the content 
learnt by the students through conventional education is sub¬ 
ject to being forgotten. This indicates a need for an approach 
which could solve the problem with retention of knowledge. 
In a problem-based process, students learn by analyzing and 
decoding a real life problem (Larsson, 2001). They generate 
hypothesis about the solution and prepare a plan to reach the 
solution. The role of the teacher is diminished to a tutor who 
usually helps with learning skills rather than content. 

The concept of problem-based assessment should be 
studied from two various perspectives. On the one hand, stu¬ 


dents’ knowledge who go through problem-based learning 
should be assessed through problem-based tests (Kwan & 
Lee, 2009), one the other hand, problem-based assessment 
targets students skills and performance rather than what they 
have accumulated in their brain. Therefore, problem-based 
assessment is considered to be a true measure of what stu¬ 
dents can really do (Savery, 2006). In problem-based as¬ 
sessment, test takers receive problem scenarios instead of 
question. They should view the problem scenario as real-life 
issue and attempt to solve it by considering the context of the 
study (Hung, 2006). 

In general studies that have accorded focus to prob¬ 
lem-based assessment have reported lower scores among 
the students in immediate posttest (Barrow, 1986; Boud & 
Feletti, 1997; Savery, 2006); however, in terms of delayed 
posttest students with problem-based education have outper¬ 
formed those in conventional classes. This issue signifies the 
importance of problem-based education. 

METHODS 
Research Design 

This study has a mixed-mode design, as it includes a quan¬ 
titative and a qualitative part. The quantitative section of the 
study has a quasi-experimental design (due to non-random 
selection of the participants through quota approach). It also 
has a within-subject design. The participants’ performance 
was measured using quantitative data (scores). The qualita¬ 
tive section of the study made use of purposive sampling 
method. Semi-structured interview questions were prepared 
by the researchers based on Hung’s (2006) 3C3R model in 
problem-based learning. 

Participants and Setting 

An intact class of master’s level students who were in the 
process of the course ‘Practical Teaching’ in the current ac¬ 
ademic semester (Semester 1, 2017) were used to pursue 
the purpose of this study. Running power analysis, the re¬ 
searcher realized that at least 33 participants are required 
for the quantitative section of the study. Gender, age, reli¬ 
gion and socio-economic background were not among the 
variables in this study and focus was accorded on partic¬ 
ipant at one particular setting- Payame Noor University. 
One of the limitations of the study is that the researchers 
could not collect data from various types of universities in 
Iran, though the course practical teaching is being taught in 
most types of universities, i.e., state, private, and distance 
universities. To solve this problem, we assumed that the 
participants in the study are similar to those in other univer¬ 
sities in the Iranian setting, as they all enter different uni¬ 
versities through the university entrance exam. Therefore, 
quota sampling cab best describe the sampling procedure 
of the study. 

As for the qualitative section of the study, 10 participants 
were selected and interviewed. As consistencies were found 
in the answers and as the researchers reached the pint of data 
saturation, no more participants were interviewed. 
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Instruments 

Three main instruments were used in this study. The first 
one was the final achievement test administered to the 
participants as the course’s final term exam. This multi¬ 
ple-choice-item test was used as the basis of comparison, in 
order to make sure the test has construct validity two experts 
in the field were asked to review the test. 

A piloted version of a researcher-made posttest was used 
to assess student-teachers’ problem-solving ability related to 
the real-life situations they encounter in the language class¬ 
es. Item analysis was conducted to check item discrimina¬ 
tion (ID) and item facility (IF) of the test items. In addition, 
construct validity of the test items was checked by a) using 
two experts in the field, b) selecting the questions based on 
the topics covered in the course book and running factor 
analysis. 

The last instrument is the semi-structured interview. The 
interview questions were selected based on the constructs of 
problem-based learning approach as stated by Hung (2006). 
This could ensure that the interview questions have construct 
validity. 

Procedure 

Consenting participants were distinguished from non-con¬ 
senting ones by distributing consent forms in order to con¬ 
duct an ethical research. The researchers collected these par¬ 
ticipants’ final term scores for the course ‘practical teaching’. 
One day after the final exam, the participants were given the 
problem-based test. The results of the problem-based test 
were compared to the final term scores (as administered by 
the university) in order to answer the first research question. 

As for the qualitative section of the study (second research 
question), Grounded Theory (GT) was used to collect samples 
of the participants utterances about the method. This inductive 
approach to qualitative data analysis is based on Corbin and 
Strauss (2008). The participants were interviewed after taking 
the problem-based test. Using the semi-structured interview, 
the researcher delved into the participants’ perception about 
the new assessment procedure, i.e. problem-based tests. 

RESULTS 

Exploratory Data Analysis Results 

As an assumption of parametric study, distribution of data 
was checked to find out whether or not parametric tools 
should be used in the study. 

As can be observed in Table 1, ratios of skewness and 
kurtosis over their perspective standards are within the range 
of +/-1.96; thus, normal distribution of scores was assumed. 

In the next stage reliability of the tests was gauged through 
Cronbach alpha formula. The results of the Cronbach alpha 
test was equal to a=.81 and a=.79 for the conventional test 
and the problem-based test respectively. 

As for the results of inter rater reliability test, there was a 
very high and significant correlation between the two raters 
who rated the samples (r =.913, p =.000); that is an accept¬ 
able agreement percentage of 83.3% between the two raters. 


The intra-rater reliability test indicated a high correlation 
(r =.898, p =.000). (Table 2). 

Comparison Between Conventional and Problem-Based 
Test Results 

The following null hypothesis was postulated based on the 
first research question: 

HO. There is no significant difference between the stu¬ 
dent-teachers’ performance on conventional university test 
and problem-based tests with regard to the course ‘Practical 
Teaching’. 

The participants scores in the conventional test was com¬ 
pared to their scores in the problem-based test using paired 
samples t-test, as the study had a within subject design. 

As can be seen in Table 3, there is difference between 
the conventional test scores (M= 22.75, SD= 3.06), and the 
problem-based test (M=12.24, SD= 12.30) 

The results of paired samples t-test (t(32)=4.78, p=.000 
<005, SD=12.61, [6.0430-14.987] indicate that the differ¬ 
ence between the scores on two tests is meaningful; therefore 
the nidi hypothesis was rejected, and it can be assumed that 
there is a significant difference between the student-teach¬ 
ers’ performance on conventional university test and prob¬ 
lem-based tests with regard to the course ‘practical teaching’. 

Perceptions About Problem-Based Tests 

The second research question of this study was: 

• What is the perception of student-teachers about prob¬ 
lem-based tests for the course “Practical Teaching”? 

In order to delve into the participants’ perception towards 
problem-based assessment, Corbin’s and Strauss’s (2008) 
Grounded Theory (GT) was used to analyze the data. GT is 
an inductive approach to data analysis. Samples of partic¬ 
ipants’ utterances were selected. Based on the samples the 
main themes were extracted. 

As for the interview questions, we made use of 3C3R 
model suggested by Hung (2006) to design the interview 
questions. This model attempts to optimize the constructs of 
problem-based learning and assessment by making sure all 
the constructs have been included in the test. The constructs 
include content, context, connection, referencing, research, 
and reasoning. Therefore, the semi structured interview 
questions were designed regarding these constructs. 

The participants believed that the questions given to them 
at the posttest well represented to the content of the course 
book. However, mostly they were unaware of the way the 
theories could be met in practicum. 

“What we read was in test, but I did not know that it 
could be the answer to the question. ” 

It seems that the participants expected to encounter a test 
which would measure their memory span; however, prob¬ 
lem-based tests assess students’ ability rather than memory. 

“/ had memorized a list of language teaching methods 
and their features. Useless for me in the test. ” 

In terms of context, the participants agreed that the prob¬ 
lem-based test was well-situated in the context of language 
classes: 
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Table 1. Distribution of the scores 



N 

Standard deviation 

Skewness 

Kurtosis 

Statistic 

Statistic 

Statistic 

Standard error 

Statistic 

Standard error 

Conventional test 

33 

3.06217 

-0.486 

0.409 

-0.476 

0.798 

Problem-based test 

33 

12.30099 

5.279 

0.409 

29.425 

0.798 

Table 2. Intra-class correlation coefficient; posttest 


Intraclass correlation 95% confidence interval 

F test with true value 0 



Lower bound Upper bound Value dfl 

df2 Sig 


Single measures 0.898 0.899 0.924 41.768 32 62 0.000 

Average measures 0.913 0.936 0.976 41.768 32 62 0.000 


Table 3. Mean score comparison 



Mean 

N 

Standard 

deviation 

Standard 

error mean 

Conventional 

test 

22.757 

33 

3.06217 

0.53306 

Problem 
based test 

12.242 

33 

12.30099 

2.14133 


“The situations in the test are exactly like real classroom 
situations. For example, selecting a technique to correct stu¬ 
dents. ” 

Some learners also mentioned more detailed features of 
the test which were observed in the test such as culture. 

“Speaking English is different from Farsi; The two cul¬ 
tures are different. For example, you talked about writing 
a letter to friend to keep my dog. This is related to Western 
culture. 1 enjoyed it. ” 

On the other hand, some participants considered contex- 
tualization a demerit and favored lesser degrees of contex- 
tualization. 

“If I did not answer the question correctly, it is because I 
did not know the context...very technical for me. ” 

Seven participants acknowledged that there was a logical 
connection between the questions and the outside world. 

“If the test was a real language class, it could test expe¬ 
rience and intelligence. ” 

As for researching, the participants believed that a 
problem-based test should not be confined to the bound¬ 
aries of conventional tests in terms of time and research 
tools. 

“We should open books to find our answers. Even in 
class, when I have problems, I ask friends. ” 

They also believed that they needed to know their goal 
before they started the search for the answer. Thus, decoding 
comes before research. 

The participants favored the collaborative nature of as¬ 
sessment in problem-based setting, and considered it a form 
of real-life assessment: 

“In real life you ask people questions; you search online. 
In this test I could do it. That is, I can express a blessing, a 
necessity. ” 


DISCUSSION 

As it was observed in this study, there is a difference between 
the results of the student-teachers’ scores in conventional 
tests and problem-based tests. The participants’ scores in the 
conventional test were significantly higher than their scores 
in problem-based test. It is generally assumed that students 
gain lower scores in problem-based tests (Barrows, 1986). 
Waters and McCracken (1997) assumed that traditional ap¬ 
proaches to assessment, e.g. multiple-choice items and true/ 
false questions cannot discriminate poor and good students 
from each other, whereas problem-based assessment distin¬ 
guishes students based on what they can do based on their 
knowledge. Indeed, conventional forms of assessment are 
accused of being narrow and scores are residt of rote mem¬ 
orization (Koschmann, Glenn, & Conlee, 1999). Many stu¬ 
dents who simply pass in conventional forms of assessment 
fail to solve the ill-structured problems in problem-based 
tests. 

Krajcik, Blumenfeld, Marx, and Soloway (2000) pointed 
out that one of the merits of problem-based tests is that they 
are contextualized. Consistent with Krajcik et al. (2000), 
Savery (2006) stated that problem-based assessment guides 
the students in certain domains. In this study, the participants 
perceived the problem-based test as contextualized; they 
stated that the questions were based on real-life situations 
that may occur in language classes. Hung (2006) argued that 
the problem scenarios should be context- and goal-specific; 
otherwise, the students will move from one context to anoth¬ 
er to find the answers. 

Problem-based assessment is a collaborative approach to 
solving problems. As a result, dealing with a large number 
of students in a test is cumbersome (Waters & McCracken, 
1997). However, Hays and Gupta (2003) noted that the col¬ 
laborative nature of PBL is one of the most significant con¬ 
structs of this method. Collaboration is a source of inquiry 
for the students. The current study dealt with rather a small 
number of students (n=33); therefore, no problem was felt in 
dealing with the participants. In addition, the participants fa¬ 
vored the collaborative nature of problem-based assessment 
and considered it an innovative approach to assessment. 

Fletti and Boud (1997) mentioned that although many 
students are more used to conventional approaches to assess¬ 
ment, they usually have low retention of knowledge. As a 
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Table 4. Paired samples t-test; conventional test vs.problem-based test 



Paired differences t 

df 

Sig. (2-tailed) 

Mean 

Standard deviation 

Standard error mean 95% confidence interval of 

the difference 





Lower Upper 



10.51 

12.612 

2.195 6.043 14.987 4.78 

32 

0.000 


result, they score higher on conventional tests but what they 
learn is subject to being forgotten. Although we did observe 
the difference in the participants’ scores in this study, we did 
not investigate the retention of knowledge among the stu¬ 
dents. One of the main reasons was that the results of prob¬ 
lem-based assessment is more accurate in case the course 
has been conducted based on problem-based approach. In 
the same vein (Kwan & Lee, 2009) asserted that problem 
learning and assessment are bi-directional in that both are 
required for successful problem-based education. 

One of the issues mentioned by the participants in the 
interviews was the connection between the questions in the 
problem-based test and with the context of the study. Hung 
(2006) assumed that connection is a significant dimension 
of problem-based learning and assessment which makes it 
meaningful. Authentic problems that students are exposed to 
in their real life are connected to the context and real life 
issues. The participants in this study acknowledged this issue 
as an issue which made solving the problems interesting for 
them. In line with this study, Hmelo-Silver (2004) stated that 
intrinsic motivation is the result raising students’ interest by 
giving them meaningful and authentic problems to solve. 

CONCLUSION 

This study attempted to solve some of the problems ob¬ 
served in conventional approaches to assessment. As stat¬ 
ed earlier in this paper, conventional assessment through 
multiple-choice items is subject to low retention. In order to 
solve this problem, more innovative approaches to assess¬ 
ment should be used. To this end, PBL was selected in this 
study. The results of the study through quantitative analy¬ 
sis revealed that although student-teachers’ scores in prob¬ 
lem-based tests are reduced significantly, they consider their 
scores a true indicator of their knowledge. In addition, the 
results of the qualitative section of the study revealed that 
in general the student-teachers have a positive perception 
about problem-based assessment and believe that conven¬ 
tional forms of assessment are not a true measure of their 
knowledge. 

This study can have pedagogical implications for test 
designers. Test designers are in charge of creating accurate 
and valid tests that can target the constructs of the course. 
It seems that previous and traditional forms of assessment 
do not assess test takers’ knowledge in terms of what they 
can do. Therefore, test takers can adopt PBL as a possible 
approach to making tests. Language teachers should also 
remind the learners of the significance of proficiency and 
skill-based learning over memorization. This can guide 
learners throughout their language learning experience. To 


achieve this goal language teachers can create authentic 
problem-based tasks in the classes and use them to increase 
learners’ proficiency. 

This issue can be subject for new research; it is signif¬ 
icant to know how PBL can affect proficiency vs. compe¬ 
tency, an issue with rare empirical evidence in the field of 
applied linguistics. It comes highly recommended by the 
researchers that the focus of future research should be on 
examining the effect of PBL on various (e.g., incidental vs. 
intentional) vocabulary learning forms. Future researchers 
may also wish to study the dynamics of collaboration among 
language learners in PBL classes. 
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